Aerial video based point, distance, and velocity real-time measurement system

ABSTRACT

A method of determining geo-reference data for a portion of a measurement area includes providing a monitoring assembly comprising a ground station, providing an imaging assembly comprising an imaging device with a lens operably coupled to an aerial device, hovering the aerial device over a measurement area, capturing at least one image of the measurement area within the imaging device, transmitting the at least one image to the ground station using a data transmitting assembly, and scaling the at least one image to determine the geo-reference data for the portion of the measurement area by calculating a size of a field-of-view (FOV) of the lens based on a distance between the imaging device and the measurement area.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority to U.S. Provisional PatentApplication Ser. No. 62/165,481, filed on May 22, 2015, and entitled“AERIAL VIDEO BASED POINT, DISTANCE, AND VELOCITY MEASUREMENT SYSTEM,”the complete disclosure of which is expressly incorporated by referenceherein.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

The invention described herein was made in the performance of officialduties by employees of the Department of the Navy and may bemanufactured, used and licensed by or for the United States Governmentfor any governmental purpose without payment of any royalties thereon.This invention (Navy Case 200,247) is assigned to the United StatesGovernment and is available for licensing for commercial purposes.Licensing and technical inquiries may be directed to the TechnologyTransfer Office, Naval Surface Warfare Center Corona Division, email:CRNA_CTO@navy.mil.

BACKGROUND OF THE DISCLOSURE

Oftentimes law enforcement and military personnel may need to obtain thegeodetic position of a point, a distance from a nominally stationary ormoving object, a distance of an impact from an absolute geodetic point,or the speed of a moving object. For example, during militaryoperations, the distance that a large caliper weapon may miss a targetcenter and, instead, impact the ocean or earth (i.e., the miss distance)is often required for training feedback and qualifications testing.Additionally, in law enforcement, it may be required to measure thespeed of traffic on a roadway or highway.

Existing systems that gather miss distances traditionally have largelogistical footprints, are expensive to acquire, expensive to operate,time consuming to deploy, and/or do not offer real-time results.Alternatively, personnel might attempt to use a small portable devicefor deployment in training areas in remote locations that are presentlydifficult to bring in large-scale equipment or that do not contain thenecessary infrastructure or land-line power nearby to operate theequipment.

SUMMARY OF THE DISCLOSURE

The present invention relates to a measurement system or scoring systemconfigured to operate in a variety of locations to measure distances,determine geodetic points, analyze speed of moving objects, and performother spatial or velocity measurements or analyses. Generally, variousembodiments of the present disclosure include an aerial video-basedpoint, distance, and velocity measurement system that orients a video orstatic capture camera underneath an aerial platform which is configuredto establish a measurement or coordinate area in which to detectgeodetic points, distance between points relative to each other, and/orthe velocity of objects moving within the area. In particular, someembodiments include systems for real-time measurements of absolutegeodetic point positions, relative distances, distances from absolutegeodetic positions, and/or vehicular velocities over a plurality ofterrain, such as flat or irregular land or water. In some embodiments, avertical camera scoring unit and/or a Global Navigation Satellite System(GNSS) may be used to measure or otherwise determine spatial distancesor velocities.

Embodiments of the present disclosure may include the use of a solitaryvideo camera operably coupled to an aerial platform (e.g., a drone,helicopter, plane, etc.), for example suspended in a nadir or plumb-boborientation via a three-axes brushless gimbal, meaning that the cameramay be oriented and face directly or straight down when coupled to theaerial device. As such, it is possible to obtain a direct overhead planview of the monitored area. For example, embodiments of this disclosurecan include a wireless or other untethered aerial platform that canmaintain a stationary or moving hover over a particular area or mayinclude a tethered or directly affixed video or other camera systemwhich can provide overhead coverage of an area but that may be poweredthrough a wired connection.

According to a first illustrative embodiment of the present disclosure,a measurement system comprises a monitoring assembly comprising a groundstation configured to receive images of a measurement area andconfigured to select a portion of the measurement area shown in theimages. The ground station comprises a scaling unit configured to scalethe images of the measurement area. The measurement system alsocomprises an imaging assembly comprising an aerial platform configuredto maintain a stationary or moving hover over the measurement area andan imaging device comprising a camera and lens operably coupled to theaerial platform, the imaging device being positioned in a nadir positionand configured to capture a plan view of the images of the measurementarea. A geodetic position of the portion of the measurement area isdetermined by the scaling unit of the monitoring assembly. Additionally,the monitoring device is configured to display an output of the absolutegeodetic position of the portion of the measurement area.

According to a further illustrative embodiment of the presentdisclosure, a method of determining geo-reference data for a portion ofa measurement area comprises providing a monitoring assembly comprisinga ground station, providing an imaging assembly comprising an imagingdevice with a lens operably coupled to an aerial device, hovering theaerial device over a measurement area, capturing at least one image ofthe measurement area within the imaging device, transmitting the atleast one image to the ground station using a data transmittingassembly, and scaling the at least one image to determine thegeo-reference data for the portion of the measurement area bycalculating a size of a field-of-view (FOV) of the lens based on adistance between the imaging device and the measurement area.

According to another illustrative embodiment of the present disclosure,a method of determining geo-reference data for a portion of ameasurement area comprises providing a monitoring assembly comprising aground station and a scaling unit and providing an imaging assemblycomprising an imaging device with a lens operably coupled to an aerialdevice, an inertial measurement unit (IMU) configured to determine anattitude (e.g. roll, pitch, yaw) of the imaging device, and a globalnavigation satellite system (GNSS) configured to determine geo-referencecoordinates of the imaging device. The method also comprises providing adata transmitting assembly comprising a telemetry consolidation unitoperably coupled monitoring assembly and the imaging assembly.Additionally, the method comprises orienting the imaging device towardan object at a known distance between the imaging device and the object,determining a size of a field-of-view (FOV) of the lens at the knowndistance, calculating a ratio of the size of the FOV relative to theknown distance, and storing the ratio in the scaling unit of themonitoring assembly. Also, the method comprises hovering the aerialdevice over the measurement area, capturing at least one image of themeasurement area with the imaging device, transmitting, with the datatransmitting assembly, the at least one image to the ground station,measuring, with the imaging assembly, a distance between the imagingdevice and the portion of the measurement area, transmitting, with thedata transmitting assembly, the distance between the imaging device andthe portion of the measurement area to the monitoring assembly, andscaling, with the scaling unit, the at least one image to determine thegeo-reference data for the portion of the measurement area using theratio stored in the scaling unit.

Additional features and advantages of the present invention will becomeapparent to those skilled in the art upon consideration of the followingdetailed description of the illustrative embodiments exemplifying thebest modes of carrying out the invention as presently perceived.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description of the drawings particularly refers to theaccompanying figures in which:

FIG. 1 is a schematic view of a measurement system of the presentdisclosure;

FIG. 2 is a schematic view of a monitoring assembly of the measurementsystem of FIG. 1;

FIG. 3a is an operational flow chart for an aerial platform of thepresent disclosure;

FIG. 3b is a schematic view of the aerial platform of the measurementsystem of FIG. 1 and operably coupled to the monitoring assembly of FIG.2 when determining a measurement of a target within a measurement orcoordinate area;

FIG. 4a is a flow chart illustrating the steps performed by themeasurement system when measuring a parameter of a target within thecoordinate area of FIG. 3b exemplifying derivation of absolute geodeticpositions (e.g., latitude/longitude);

FIG. 4b is as schematic view of a camera of the measurement system ofFIG. 1 taking a measurement of a stationary target and a point of impactof a projectile or other device that was intended for the target withinthe coordinate area exemplifying derivation of absolute geodeticpositions (e.g., latitude/longitude);

FIG. 4c is a further flow chart illustrating computer functions, memoryconstructs, and user interactions needed to generate an absolutegeodetic score;

FIG. 5a is a flow chart illustrating the steps performed by themeasurement system when measuring relative miss distance between atarget within the coordinate area and the point of impact of aprojectile or other device that was intended for the target;

FIG. 5b is a schematic view of the camera of the measurement system ofFIG. 1 taking a measurement of the relative miss distance within thecoordinate area;

FIG. 5c is a further flow chart illustrating computer functions, memoryconstructs, and user interactions needed to generate a relative distancemeasurement or score;

FIG. 6a is as flow chart illustrating the steps performed by themeasurement system when measuring the velocity of a moving object withinthe coordinate area;

FIG. 6b is a schematic view of the camera of the measurement g system ofFIG. 1 taking a measurement of the velocity of the moving object withinthe coordinate area;

FIG. 6c is a further flow chart illustrating computer functions, memoryconstructs, and user interactions needed to generate a velocitydetermination; and

FIG. 7 is a schematic view of a camera lens of the camera of themeasurement system of FIG. 1 undergoing a calibration process.

DETAILED DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention described herein are not intendedto be exhaustive or to limit the invention to precise forms disclosed.Rather, the embodiments selected for description have been chosen toenable one skilled in the art to practice the invention.

Referring to FIG. 1, a measurement system 10 includes a monitoringassembly 12 which may contain a computer and monitor (e.g., a groundstation) for marking measurement points and computing the distances orscores and a vertical camera scoring assembly 100 which includes: animaging assembly 14; a data transmitting assembly 16; a videotransmitter 128 and antenna 131; a GNSS 118 and antenna 125; an aerialplatform 18 with a propulsion device (e.g., rotors, propellers, etc.) toprovide overhead images and telemetry data to monitoring assembly 12. Inone illustrative embodiment, the propulsion device may comprise multiplerotors (e.g., quadrotors) operably coupled to the aerial platform 18.Exemplary aerial platforms are available from AerialPixels, SZ DJITechnology Co. Ltd. and from Parrot, and are described in U.S. Pat. No.9,321,530 to Wang et al. and U.S. Pat. No. 8,474,761 to Callou, thediscloses of which are expressly incorporated herein by reference.

Any of the components of measurement system 10 may be powered by anon-board or a remote power supply 150, such as lithium polymerbatteries, or may be powered by a wired connection (e.g., an electricalwire or cable) to a power source. Measurement system 10 is configuredfor overhead measurement of a coordinate or measurement area 32 on aground surface G. In one embodiment, ground surface G may be anygenerally flat terrain on land or may be water. Measurement system 10 isconfigured to measure, detect, or otherwise determine a geodeticposition of a stationary or moving object or target within coordinatearea 32, a velocity of moving object or target within coordinate area32, and/or a distance between an object within coordinate area 32 andthe impact of a projectile or other object intended for the object(“miss distance”). Additionally, in one embodiment, measurement system10 is configured to detect, measure, or otherwise determine a geodeticposition of a virtual object, such as a spot on an ocean surface that isbeing targeted that lacks a physical real-world target.

Referring still to FIG. 1, in one embodiment, imaging assembly 14includes a camera 106 with a lens 103, for capturing static and/ormoving images (i.e., video). Illustratively, lens 103 may be alow-distortion rectilinear lens. Lens 103 is configured with a left-mostline of sight 20 and a right-most line of sight 22 which defines afield-of-view (FOV) 24 of lens 103 such that camera 106 is configured tocapture any moving or stationary objects on ground surface G withinfield-of-view (FOV) 24. The entire FOV 24 of camera 106 and lens 103 iscomprised of a fore/aft dimension that, along with the left and rightlines of sight 20, 22, can define a two-dimensional FOV area of camera106 that intercepts ground surface G (e.g., land, water).Characteristics of the lens 103 define the FOV 24. Imaging assembly 14also may include a distance measuring device, such as a laserrangefinder 104, operably coupled to camera 106 to detect, measure, orotherwise determine the distance from camera 106 to ground surface G. Inone embodiment, distance measuring device is comprised of rangefinder104, a laser rangefinder or lidar, an altimeter, or any other deviceconfigured for distance measurement between ground G and camera 106.

As shown in FIG. 1, imaging assembly 14 also includes an inertialmeasurement unit (IMU) 109 for determining the horizontal and/orvertical attitude (e.g., pitch, roll, yaw) of camera 106. Moreparticularly, the position of camera 106 and, therefore lens 103, may bemoved or adjusted based on data sensed, measured, or otherwisedetermined by IMU 109. Imaging assembly 14 or any other assembly ofmeasurement system 10 also may include a Global Navigation SatelliteSystem (GNSS) 118 configured to detect or otherwise determine thegeodetic coordinate or positions (e.g., longitude, latitude, and/oraltitude) of camera 106. GNSS 118 may include an antenna 125 or othertransmitting/receiving member for detecting global coordinates of camera106 through satellite, cellular, or any other data transmissions. Itshould be appreciated that there may be an offset (x, y and/or zcoordinates) between the GNSS 118 and the camera 106, thereby requiringa translation between GNSS position and camera position.

Referring still to FIG. 1, in one embodiment, IMU 109 is coupled to asupport assembly 26 of measurement system 10. Illustratively, supportassembly 26 may define a three-axis brushless gimbal configured to movecamera 106 and rangefinder 104 in any of the x, y, and z axes. Movementof support assembly 26 may be electronically and/or automaticallycontrolled through a gimbal control unit (GCU) 34. Support assembly 26may include a plurality of movable devices 28, for example rotatablebrushless motors, pulleys, or rotatable bearings. In one embodiment,support assembly 26 stabilizes camera 106 in a straight down positionsuch that lens 103 is maintained in a plumb bob or nadir position tocapture plan views of coordinate area 32.

As shown in FIG. 1, GCU 34 is coupled to moveable devices 28 and can beprogrammed in advance to maintain camera 106 and rangefinder 104 in astable nadir position, instantaneously compensating for movement ofaerial platform 18.

Once camera 106 captures a static or video image of coordinate area 32,images and other data (e.g., telemetry or position data) from camera106, IMU 109, GNSS 118, and rangefinder 104 may be transmitted to datatransmitting assembly 16. In one embodiment, data transmitting assembly16 is hard-wired to imaging assembly 14, however, in an alternativeembodiment, data transmitting assembly 16 may be wirelessly coupled toimaging assembly 14 through a cellular, Bluetooth®, wifi, satellite, orany other wireless connection. In certain illustrative embodiments, adata security system may be incorporated within data transmittingassembly 16 to protect data transmission. For example, an encryptionprotocol (such as Wired Equivalent Privacy (WEP2)) may be stored on awifi card coupled to a bus system that communicates between the camera106 and the monitoring assembly 12.

As shown in FIG. 1, data transmitting assembly 16 includes a telemetryconsolidation unit 121, a transmitter 129 operably coupled to telemetryconsolidation unit 121, and an antenna 127 operably coupled totransmitter 129. Telemetry consolidation unit 121 is configured toreceive and/or transmit data from/to imaging assembly 14 and monitoringassembly 12 such that the data obtained by imaging assembly 14 istransmitted to monitoring assembly 12, such as the images obtained bycamera 106 and the distances obtained by rangefinder 104.

Referring to FIG. 2, monitoring assembly 12 may be configured to bothreceive and transmit data from/to imaging assembly 14 through datatransmitting assembly 16. In one embodiment, monitoring assembly 12includes a computer 40 with a memory and a processor configured toexecute machine-readable instructions. In one embodiment, computer 40may be portable, such as a laptop or tablet-style device. Monitoringassembly 12 may define a ground station for measurement system 10 suchthat a user can control measurement system 10 from a remote location andcompute measurements or scores from a location remote to aerial platform18 and camera 106. Computer 40 includes a display or monitor 42 and atleast one input device 44. For example, input device 44 may be analphanumeric keyboard, a mouse, a joystick, stylus, or any other type ofdevice configured to provide an input from a user to computer 40.Computer 40 is operably coupled to a power supply 46, for example abattery or external power source.

Computer 40 either includes or is operably coupled to a plurality ofsoftware modules such as scaling unit 48 and a scoring unit 50.Additionally, computer 40 is operably coupled to camera 106 of imagingassembly 14. Because camera 106 has both video and static imagingcapabilities, camera 106 may include or be operably coupled to a videoreceiver 52 of imaging assembly 14 which includes an antenna 53. Becausecamera 106 has video capabilities, the user may use video commands, suchas pause, play, fast-forward, reverse, and slow-motion to watch anyportion of the video obtained from camera 106. Additionally, camera 106and video receiver 52 may be operably coupled to scaling unit 48.Computer 40 also may be coupled to a telemetry data receiver 54 of datatransmitting assembly 16 which includes an antenna 56. In oneembodiment, video receiver 52 and video antenna 53 are operably coupledto a video transmitter 128 and video antenna 131 and/or monitoringassembly 12. Additionally, telemetry data receiver 54 and antenna 56 areoperably coupled to monitoring assembly 12 and imaging assembly 14.

Referring to FIGS. 3a and 3b , in operation, measurement system 10 maybe used to determine the geodetic position of a static target orlocation within a particular area and also may be used to determine amiss distance of an impact of a projectile away from a target and/or todetermine the velocity of a moving object or target within a particulararea. More particularly, at least camera 106 of imaging assembly 14 iscoupled to aerial platform 18. Illustratively, aerial platform 18 is anytype of device configured to fly or hover above ground surface G. Forexample, aerial platform 18 may be any unmanned aerial vehicle (“UAV”)or device, such a drone or remote-controlled airplane, helicopter, orother similar device configured to be remotely operated. Alternatively,aerial platform 18 may be a manually-controlled aerial vehicle or astatic structure, such as a bridge or tower. Aerial platform 18 may bepowered by lithium polymer batteries. In one embodiment, GNSS 118 andthe propulsion system for aerial device 18 cooperate to define aposition maintenance system for maintaining the position of aerialdevice 18 during a stationary hover. For example, GNSS 118 can determineif the position of aerial device 18 is shifting/drifting, and becauseGNSS 118 and the propulsion system are hard-wired to aerial device 18,GNSS 118 and the propulsion system work together through hard-wiredcommunication to maintain the position of aerial device 18.

As aerial platform 18 flies or hovers or moves over a particular area,coordinate area 32 may be defined on ground surface G by FOV 24 of lens103. In one embodiment, aerial platform 18 is configured to hover overcoordinate area 32 at a location which contains a target 1000 thereinwhich may be moving or stationary. As shown in FIG. 3, target 1000 maybe a virtual target at a geodetic longitudinal and latitudinal positon.Camera 106 is configured to capture live static or video images of thestationary or moving object and transmit images via video transmitter128 and video antenna 131 and data (e.g., time tags, coordinateinformation, etc.) to monitoring assembly 12 where it is received byvideo receiver 52 and antenna 53. For example, data such as the positionof camera 106 may be obtained from GNSS 118 and/or IMU 109 andtransmitted through data transmitting assembly 16 to monitoring assembly12. Data regarding the image time stamp of each frame of the video canbe similarly transmitted to monitoring assembly 12. Data fromrangefinder 104 also may be transmitted to monitoring assembly 12through data transmitting assembly 16. Data from GNSS 118, IMU 109, andRangefinder 104 will provide a continuous plurality of measurements withtime tag associations at an update rate corresponding to or greater thanthe frame rate of to the video or static images. With this data,monitoring assembly 12 may employ scaling unit 48 and/or scoring unit 50to provide images, text, or any other type of output to convey theinformation (e.g., geodetic point positions, distances, scores,velocities, and/or recorded video) being captured by measurement system10 to a user.

FIG. 3a discloses an exemplary operational chart for operating aerialplatform 18 from computer 40. For example, at a Step 300, lenscalibration is performed to derive constants (see FIG. 7) and the lenscalibration constants are stored in the non-volatile memory of groundstation 40. In Step 301, GCU 34 is programmed to perform the movementand stabilization of camera 106 and rangefinder 104 via a 3 axesbrushless gimbal in a nadir or straight down position at all times.Accordingly, the nadir position will be instantaneously adjusted forirrespective of roll, yaw, or pitch movements of aerial platform 18. Ata Step 302, computer 40 may be turned on via a power supply 46, such asbatteries. At a Step 304, aerial platform 18 also may be turned on via apower supply 150. At a Step 304, all on-board devices, such as telemetryconsolidation unit 121, on aerial platform 18 may be turned on and willdraw power from the power supply 150 powering aerial platform 18. In aStep 305, camera 106 is turned on via power supply 150.

In a Step 306, a user may open a scoring and/or scaling program orsoftware from a memory of computer 40 using user input 44. In a Step308, communication may be established between computer 40 and aerialplatform 18, for example may be hard-wired through electrical wires,cables, or tethers or may be wirelessly connected through a radiofrequency, satellite, BLUETOOTH, or other wireless network. In a Step310, geographic coordinates, data, or positions may be provided to GNSS118 from computer 40 to determine where aerial platform should fly to orhover over to capture images of coordinate area 32. In a Step 312, thepropulsion system of aerial device 18 may be activated. In a Step 314,aerial platform 18 flies to the geographic location and, in a Step 316,hovers (moving or stationary) over coordinate area 32. In a Step 318,camera 106 obtains images of coordinate area 32. In a Step 320, thelocation of camera 106 may be obtained from GNSS 118. In a Step 322, theimages and/or data from camera 106 are transmitted to video receiver 52through a wireless connection on aerial platform 18. In a Step 324, datais transmitted from camera 106 to telemetry data receiver 54 via videotransmitter 128 and from GNSS 118 to telemetry data receiver 54. In aStep 326, the scaling and scoring process is initiated on ground station40 according to any of FIGS. 4a-4c, 5a-5c, and/or 6a-6d . In a Step 328,aerial platform 18, computer 40, and/or the propulsion device arepowered off.

Referring to FIGS. 4a-4c , measurement system 10 is illustrativelyconfigured to operate according to an absolute scoring process 400 todetermine an absolute score related to the geodetic position of anon-physical object, such as a virtual target 1000, within coordinatearea 32. For example, a virtual target 1000 may be a geodeticlatitudinal/longitudinal point upon ground surface G. More particularly,using monitoring assembly 12, a user is able to see coordinate area 32through lens 103 of camera 106 which transmits live or real-time imagesof the current location of camera 106 and of coordinate area 32 in Step402 of absolute scoring process 400. In Step 404, when the area aroundtarget 1000 is flown over by aerial platform 18 (e.g., coordinate area32), the user sees coordinate area 32 on monitor 42 of computer 40because the image(s) obtained by camera 106 are transmitted to computer40 through video transmitter 128. For example, camera 106 may transmitimages with video transmitter 128 and video antenna 131 which arereceived by video antenna 53 of video receiver 52 which allows imagesfrom camera 106 to be shown on monitor 42. Camera 106 is maintained in anadir position through the three-axis gimbal with GCU 34. Telemetry datareceiver 54 communicates with telemetry consolidation unit 121 totransmit data from IMU 109, GNSS 118, and/or rangefinder 104 tomonitoring assembly 12.

More particularly, telemetry consolidation unit 121 containsmicrocontrollers to read and time tag the data from rangefinder 104,GNSS 118, and IMU 109 and consolidates such data into a single datastream. The consolidated data stream is then transmitted to monitoringassembly 12 via transmitter 129 and antenna 127. For example, datastreams from rangefinder 104, GNSS 118, and IMU 109 are merged togetherby telemetry consolidation unit 121 with time and/or data stamps or tagsfor individual data samples and the time tags also can be synchronizedto video data frames generated by camera 106. In one embodiment, a planview of the images/video obtained from camera 106 are shown on monitor42, however, in an alternative embodiment, three-dimensional images areshown on monitor 42.

Subsequently, in Step 406, using input device 44, the user moves acursor or other indicator on monitor 42 over coordinate area 32 andclicks to select target 1000 during a “Get Cursor Position” step. Uponclicking or marking a point (e.g., a target 1000) within coordinate area32, which may be a point of impact, a stationary object, or any otherfixed location within coordinate area 32, the location of is identifiedby geodetic coordinates within monitoring assembly 12. Illustratively,as shown in FIG. 4b , the geodetic latitudinal and longitudinal positionof camera 105, as held in the nadir or plumb-bob position by supportassembly 26, is the same geodetic latitudinal and longitudinal positionas virtual target 1000, which lies directly underneath camera 106 and isdepicted by vertical line or centerline/bore line of sight 58.Therefore, the location of virtual target 1000, as shown in FIG. 4b ,may be the same as reported by GNSS 118. Additionally, an absolute scoreof a splash 2000, which is offset from virtual target 1000 in FOV 24 andcoordinate area 32 may be determined.

In Step 406, the images/data shown on monitor 42 can be divided into aplurality of equal units. For example, in one embodiment, monitoringassembly 12 and/or data transmitting assembly 16 may be configured todivide or apportion the images and data shown on monitor 42 into 1000 ormore equal units, each having a length extending along the x and y axes(i.e., the respective left/right and top/bottom directions of monitor42). Input device 44 is moveable by the operator along the x and y axesto move a cursor or other indicator shown on monitor 42. This equalapportionment of x and y coordinates upon monitor 42 allows precisescreen distances to be measured for later scaling. Whenever the operatorclicks or otherwise provides an input to input device 44 to indicate aposition upon the image shown on monitor 42, the position of the cursoror indicator along the x and y axes or the apportioned box or unit inwhich the user marked target 1000 is stored by monitoring assembly 12.Monitoring assembly 12 also provides an option to the operator tofast-forward, reverse, pause, slow down, or speed up any video obtainedfrom camera 106 and shown on monitor 42. By allowing for these optionswith video data, the operator may accurately identify a point incoordinate area 32 by freezing the frame at a moment when the positionis required, or in the case of a scoring impact of a large gun on wateror land, which creates a splash or puff of smoke, the operator, forgreater accuracy, could reverse the video back in time to the videoframe that brings the splash/puff of smoke to a small point.

In Step 408, during the scaling procedure or a “Computer Score” step anda “Scale Video Computation” step, the images and/or video shown oncomputer 40 and obtained via video transmitter 128 and video antenna 131from camera 106 may be scaled by using the x and y screen coordinates inconjunction with information about FOV 24 from lens 103. The exemplaryvideo can be scaled by scaling unit 48. For example, scaling unit 48 mayuse predetermined and stored values of constants or parameters of FOV 24of lens 103 and a distance from ground surface G as provided byrangefinder 104 to compute a size of FOV 24 at any given rangedetermined by rangefinder 104. More particularly, scaling unit 48 isconfigured to apply a ratio of the distance between camera 106 and anobject relative to the length or size of FOV 24 of lens 103 and, usingthat ratio which may be stored and predetermined from previous testingof camera 106 and knowing the distance between camera 106 and target1000 (determined, for example, by rangefinder 104), scaling unit 48 isconfigured to determine the size of FOV 24 so the operator understandsthe distances and parameters shown on monitor 42. For example, thescreen distance from virtual target 1000 to splash 2000 can bedetermined from the x and y screen coordinates that an operator may markwith respect to the x and y coordinates of virtual target 1000. The xand y screen coordinates of virtual target 1000 correspond to the exactcenter of a video image displayed on display 42 because that is theboresight of the video image. An actual distance between virtual target1000 and splash 2000 is represented by vector 43 in FIG. 4b . Thedistance of vector 43 may be computed as a ratio of FOV 24 at a givenrange by rangefinder 104 to the screen distance of vector 43. Vector 43is the offset vector from the boresight of camera 106 which is computedwith forward geodetic computation.

In Step 410, and using scoring unit 50 of monitoring assembly 12, anabsolute score is determined which identifies an absolute geodeticlocation of an impact point or a geodetic point, splash 2000, or offsetfrom virtual target 1000 within coordinate area 32. The absolute scoreof virtual target 1000 is computed using the latitudinal andlongitudinal positions (geodetic position) and north bearing of theimage of camera 106, as determined by GNSS 118 and/or IMU 109,respectively, which, therefore, identifies the coordinates and geodeticposition of camera 106. More particularly, using the scaled distances ofvector 43 and derived angle of vector 43, as gleaned from GNSS 118 andIMU 109, an absolute geodetic location of splash 2000 is calculatedusing forward geodetic calculations. The absolute score of target 1000identifies distances and/or the angular offset of splash 2000 relativeto virtual target 1000 and/or any other object. In this way, theabsolute score calculates the geodetic position of splash 2000 which maybe displayed on monitor 42. Moreover, a virtual target such as virtualtarget 1000 does not necessarily have to be positioned directlyboresight beneath camera 106 as illustrated in FIG. 4b . Those skilledin the art will realize that further combinations of the steps outlinedabove may be used with the forward and inverse geodetic calculations tocompute the coordinates of splash 2000 relative to a target offset fromcenterline position 58 of target 1000, as illustrated in FIG. 4 b.

In Step 412, the absolute score of target 1000 is provided to monitor 42for the operator to review the output of Step 410. More particularly,using scoring unit 50, data is transmitted to monitor 42 to provide agraphical, textual, pictorial, or any other visual output to theoperator to understand the geodetic positions of target 1000 and splash2000. The absolute score of target 1000, as well as the data receivedfrom imaging assembly 14 to calculate the absolute score using scoringunit 50, may be saved on monitoring assembly 12 for future review orreference, to assist with subsequent calculations or comparisons of dataobtained by camera 106, etc.

Also in process 400, the IMU's angular yaw orientation data is alreadyused to compute absolute latitude/longitude with the forwardcomputation. All data during the “Compute Score” step is taken at thetime of impact. Further, during the “Scale Video Computation” step,distance offsets from latitude and longitude of aerial platform 18 areobtained as projected on ground surface G to impact point usingrangefinder 104 and lens 103 stored FOV 24 constants to determine FOV 24span at ground surface G. All data during the “Scale Video Computation”step is taken at the time of impact.

Moreover, FIG. 4c illustratively details a computer pseudocode of thefunctions, memory constructs, flow of code, and user interaction throughinput 44 that computer 40 would execute at ground station 12 to generatean absolute geodetic measurement or score.

Alternatively, as shown in FIGS. 5a-5c , measurement system 10 isconfigured to operate according to a relative scoring process 500 todetermine a relative score related to the miss distance withincoordinate area 32 between the position of an object, such as target1000, and the position of a point of impact of projectile, ammunition,weapon, or any other device intended to contact target 1000.

First, in Step 502, using monitoring assembly 12, the operator is ableto see coordinate area 32 through lens 103 of camera 106 which transmitslive images of the current location of camera 106 and of coordinate area32. In Step 504, when target 1000 is shown in coordinate area 32, theoperator sees target 1000 on monitor 42 of computer 40 because theimage(s) obtained by camera 106 are transmitted to computer 40 throughvideo transmitter 128 and video antenna 131. For example, camera 106 maytransmit images which are received by antenna 53 of video receiver 52and video receiver 52 communicates with video transmitter 128 totransmit the images or data from camera 106 to video receiver 52 throughantenna 53. Telemetry data receiver 54 communicates through antenna 56with telemetry consolidation unit 121 to transmit data from IMU 109,GNSS 118, and/or rangefinder 104 to monitoring assembly 12 through datatransmitting assembly 16.

More particularly, telemetry consolidation unit 121 containsmicrocontrollers to read and time tag the data from the rangefinder 104,GNSS 118, and IMU 109 and consolidates such data into a single datastream. The consolidated data stream is then transmitted to monitoringassembly 12 via transmitter 129 and antenna 127. For example, datastreams from rangefinder 104, GNSS 118, and IMU 109 are merged togetherby telemetry consolidation unit 121 with time and/or data stamps or tagsfor individual data samples and the time tags also can be synchronizedto video data frames generated by camera 106. In one embodiment, a planview of the images/video obtained from camera 106 are shown on monitor42, however, in an alternative embodiment, three-dimensional images areshown on monitor 42.

In Step 506, once target 1000 is identified on monitor 42, the operator,using input device 44, initiates a “Get Cursor Position” step by movinga cursor or other indicator on monitor 42 to target 1000 whichidentifies target 1000. The operator may click on target 1000 toinitiate relative scoring process 500. and computer 40 records thecursor positions of both target 1000 and point of impact 60, asdisclosed herein. Upon clicking or otherwise marking or identifyingtarget 1000, an indicator, such as an “X”, may be visually shown onmonitor 42.

Either prior to or after Step 506, and using input device 44, theoperator moves the cursor or other indicator on monitor 42 to a splashor puff of smoke or dirt which identifies a point of impact 60 in Step508. More particularly, upon clicking or marking point of impact 60, anindicator, such as an “X”, may be visually shown on monitor 42 so thatuser can see point of impact 60.

Referring to Step 510, a “Compute Score” step, the images and/or videoshown on computer 40 and obtained from camera 106 via data transmittingassembly 16 may be scaled so the operator at monitor 42 understands therelative distances and locations of any object(s) within coordinate area32. The exemplary video can be scaled by scaling unit 48. in “ScaleVideo Computation” step. For example, scaling unit 48 may usepredetermined and stored values of constants or parameters of FOV 24 oflens 103 and a distance from ground surface G as provided by rangefinder104 (FIG. 1) to compute a size of FOV 24 at any given range determinedby rangefinder 104. This scaling procedure of the distance upon thescreen, i.e., the difference in the x and y screen coordinates betweentarget 1000 and splash 2000 and the scaling of FOV 24 of camera 106equipped with lens 103, as detailed below in FIG. 7, is thesubstantially the same as described above for FIGS. 4a and 4 b.

In Step 512, and using scoring unit 50 of monitoring assembly 12, arelative score is determined which identifies the location of target1000 and the location of point of impact 60 within coordinate area 32relative to each other. Using the scaled distances determined in Step510, a distance between target 1000 and point of impact 60 is calculatedor otherwise determined by scoring unit 50. In this way, the missdistance between target 1000 and point of impact 60 is displayed onmonitor 42 which provides the operator with information to understandhow and where the equipment, projectile, ammunition, or other devicemissed hitting target 1000. As shown in FIG. 5b , a vector drawn fromtarget 1000 to impact 60 is shown as 55, that can represent the missdistance and the angle of impact miss from target 1000. Moreover, byusing the angle of the video from camera 106, that may be found fromdata from IMU 109, the angle of vector 55, with respect to theline-of-fire, accounting for the alignment of camera 106, could becalculated and displayed on monitor 42. Vector 55 from target 1000 topoint of impact 60 may be scaled during the “Scale Video” Computation”Step and the direction of vector 55 can be adjusted to absolute or truenorth if desired.

In Step 514, the miss distance between target 1000 and point of impact60 is provided to monitor 42 for the operator to review the output ofStep 512. More particularly, using scoring unit 50, data is transmittedto monitor 42 to provide a graphical, textual, pictorial, or any othervisual output to the operator to understand the relative positions oftarget 1000 and point of impact 60 and the miss distance therebetween.The relative score of the miss distance between target 1000 and point ofimpact 60, as well as the data received from imaging assembly 14 tocalculate the relative score using scoring unit 50, may be saved onmonitoring assembly 12 for future review or reference, to assist withsubsequent calculations or comparisons of data obtained by camera 106,etc.

Moreover, FIG. 5c illustratively details a computer pseudocode of thefunctions, memory constructs, flow of code, and user interaction throughinput 44 that computer 40 would execute at ground station 12 to generatea relative distance measurement or score.

In a further embodiment, as shown in FIGS. 6a-6c , measurement system 10is configured to operate according to a speed scoring process 600 todetermine a velocity or speed of an object, such as target 1000, movingwithin/across coordinate area 32.

First, in Step 602, using monitoring assembly 12, the operator is ableto see coordinate area 32 through lens 103 of camera 106 which transmitslive images of the current location of camera 106 and of coordinate area32. In Step 604, when target 1000 is shown in coordinate area 32, theoperator sees target 1000 on monitor 42 of computer 40 because theimage(s) obtained by camera 106 are transmitted to computer 40 throughvideo transmitter 128 and video antenna 131. For example, camera 106 andaerial platform 18 hover steadily when speed scoring process 600 isinitiated in order to transmit images of coordinate area 32 which arereceived by antenna 53 of video receiver 52 and video receiver 52communicates with video transmitter 128 to transmit the images or datafrom camera 106 to video receiver 52 through antenna 53 which allowsimages from camera 106 to be shown on monitor 42. Telemetry datareceiver 54 communicates with telemetry consolidation unit 121 totransmit data from IMU 109, GNSS 118, and/or rangefinder 104 tomonitoring assembly 12.

More particularly, telemetry consolidation unit 121 containsmicrocontrollers to read and time tag the data from rangefinder 104,GNSS 118, and IMU 109 and consolidates such data into a single datastream. The consolidated data stream is then transmitted to monitoringassembly 12 via transmitter 129 and antenna 127. For example, datastreams from rangefinder 104, GNSS 118, and IMU 109 are merged togetherby telemetry consolidation unit 121 with time and/or data stamps or tagsfor individual data samples and the time tags also can be synchronizedto video data frames generated by camera 106. In one embodiment, a planview of the images/video obtained from camera 106 are shown on monitor42, however, in an alternative embodiment, three-dimensional images areshown on monitor 42.

In Step 606, once target 1000 is identified on monitor 42, the operatorduring a “Get Cursor Position” step, using input device 44, moves acursor or other indicator on monitor 42 to target 1000 which identifiestarget 1000 just as target 1000 enters coordinate area 32 at a time Ti.For example, if target 1000 is a moving vehicle, the operator watchescoordinate area 32 and/or a surrounding area captured by camera 106 andidentifies target 1000 at a certain point (e.g., a line or otheridentifier within coordinate area 32). The operator may click on target1000 to initiate speed scoring process 600. Upon clicking or otherwisemarking or identifying target 1000, an indicator, such as an “X”, may bevisually shown on monitor 42.

After Step 606, and using input device 44 in Step 608, the operatormoves the cursor or other indicator on monitor 42 to the location oftarget 1000 when target 1000 leaves coordinate area 32 at a time Ti.More particularly, upon clicking or marking the point when target 1000leaves coordinate area 32, an indicator, such as an “X”, may be visuallyshown on monitor 42 so that user can see both the point of entry and thepoint of exit for target 1000.

Referring to Step 610, the images and/or video shown on computer 40 andobtained from video transmitter 128 via camera 106 may be scaled so theoperator at monitor 42 understands the relative distances and locationsof any object(s) within coordinate area 32 during a “Computer ScoreDistance” step and a “Scale Video Computation” step. The exemplary videocan be scaled by scaling unit 48. For example, scaling unit 48 may usepredetermined and stored values of constants or parameters of FOV 24 oflens 103 and a distance from ground surface G as provided by rangefinder104 (FIG. 1) to compute a size of FOV 24 at any given range determinedby rangefinder 104. This scaling procedure of the distance upon thescreen, i.e., the difference in the x and y screen coordinates as itenters and exits FOV 24 of camera 106 and the scaling of FOV 24 ofcamera 106 equipped with lens 103, as detailed below in FIG. 7, is thesubstantially the same as described above for FIGS. 4a and 4 b.

In Step 612, and using scoring unit 50 of monitoring assembly 12, arelative score is determined which identifies the velocity of target1000 within coordinate area 32. Using the scaled distances determined inStep 610, the distance travelled by target 1000 within coordinate area32 is calculated or otherwise determined by scoring unit 50.Additionally, scoring unit 50 uses time stamp data to calculate thelength of time it took target 1000 to travel the calculated distancewithin coordinate area 32. In this way, using the standard equation ofvelocity=distance/time, the velocity of target 1000 is calculated.

In Step 614, the velocity of target 1000 is provided to monitor 42 forthe operator to review the output of Step 612. More particularly, usingscoring unit 50, data is transmitted to monitor 42 to provide agraphical, textual, pictorial, or any other visual output to theoperator to understand the velocity of target 1000 within coordinatearea 32. The speed score of target 1000, as well as the data receivedfrom imaging assembly 14 to calculate the speed score using scoring unit50, may be saved on monitoring assembly 12 for future review orreference, to assist with subsequent calculations or comparisons of dataobtained by camera 106, etc.

Moreover, FIG. 6c illustratively details a computer pseudocode of thefunctions, memory constructs, flow of code, and user interaction throughinput 44 that computer 40 would execute at ground station 12 to generatea velocity determination.

Referring to FIG. 7, a derivation of lens calibration constants neededfor scoring processes 400, 500, 600 are shown. For example, lenscalibration constants may be a width of FOV 24 and a given distance 70to camera 106. As shown in FIG. 7, FOV 24 may project verticallydownward, as shown in FIG. 1, or project horizontally, as shown in FIG.7, depending on the orientation of camera 106. Regardless of theorientation of lens 103 and camera 106, in one embodiment, centerline orbore line of sight 58 of lens 103 is configured to intersect coordinatearea 32 at a 90-degree angle in a surveyed area of coordinate area 32.FOV 24, for example, could be set against a perfectly straight concretewall in order to make as geometrically perfect measurement of FOV 24 aspracticable. Distance 70 between lens 103 and coordinate area 32 and theparameters of FOV 24, including left-most and right-most lines of sight20, 22, may be stored within monitoring assembly 12 for use duringscoring processes 400, 500, 600. The various embodiments disclosedherein include lens 103 which is of a rectilinear type providing alinear scaling when conjoined with camera 106 with digital imagingcapability. However, if lens 103 is not entirely linear, a compensatorycurve fit to the non-linearity of lens 103 could be derived from thelens calibration range depicted in FIG. 7 by scoring/measuring regularlyand surveyed spaced visual targets, within the video image, along FOV24.

Additionally, a ratio of distance derived from rangefinder 104 todistance 70 allows computation of FOV 24 of camera 106 at a distancevalue provided by rangefinder 104 at a score time. For example, tocalibrate lens 103, camera 106 may be oriented horizontally, as shown inFIG. 7, for easy access to camera by a user. With camera 106 positionedhorizontally, lens 103 faces a target, for example a wall, at a knowndistances from camera 106. Then, the size of FOV 24 of lens 103 may becalculated based on the length, height, or other distance that FOV 24captures on the target. As such, a ratio is developed which relates thesize of FOV 24 based on the distance of camera 106 to the target (e.g.,a wall). This known ratio then may be stored as a data point orparameter of camera 106 and lens 103 such that the same ratio may beapplied to any of scoring processes 400, 500, 600. More particularly,because rangefinder 104 determines the known distance of camera 106 fromground surface G, the ratio determined in FIG. 7 can be used tocalculate the instantaneous size of FOV 24 based on the measurementdetermined from rangefinder 104 when camera 106 is in a verticalorientation during field operation. By knowing the size of FOV 24 whencamera 106 is used in the field, as it is instantaneously varying due tofluctuations in the vertical positioning of aerial platform 18, thepositions of target 1000 and/or distances within coordinate area 32 maybe determined. Screen coordinate distances can be determined by the userinto computer 40 and monitor 42 through clicking on the target andimpact positions as previously described during scoring processes 400,500, 600. These screen coordinate distances can be scaled against thesize of the derived instantaneous FOV 24, as ratios of the full FOV, asshown on the full extents of the video image upon the monitor 42, to thepartial FOVs represented by the screen coordinate distances.Accordingly, distances along ground, G, as shown in the plan view imageare determined.

As may be appreciated, the system 10 is configured to propel an imagingassembly 14 to a desired height above a target, maintain the position ofthe imaging assembly 14 relative to the target, maintain a nadirposition of a cameral of the imaging assembly, transmit image andtelemetry data from the imaging assembly and a base station, scale theimage based upon characteristics of the camera lens, and provide scoringinformation based upon distance between a target impact point and anactual impact point in the scaled image.

Although the invention has been described in detail with reference tocertain preferred embodiments, variations and modifications exist withinthe spirit and scope of the invention as described and defined in thefollowing claims.

The invention claimed is:
 1. A method of determining geo-reference data,a relative distance, and a velocity of a target within a measurementarea, comprising: providing a monitoring assembly comprising a groundstation and a scaling unit, and the ground station including a displaydevice and an input device operably coupled to the display device, andthe input device is configured to receive an input from a user toidentify at least one target displayed on the display device; providingan imaging assembly comprising an imaging device with a lens operablycoupled to an aerial device, an inertial measurement unit (IMU)configured to determine movement of the imaging device, and a globalnavigation satellite system (GNSS) configured to determine geo-referencecoordinates of the imaging device; providing a data transmittingassembly comprising a telemetry consolidation unit operably coupledmonitoring assembly and the imaging assembly; orienting the imagingdevice toward an object at a known distance between the imaging deviceand the object; determining a size of a field-of-view (FOV) of the lensat the known distance; calculating a ratio of the size of the FOVrelative to the known distance; storing the ratio in the scaling unit ofthe monitoring assembly; hovering or flying the aerial device over themeasurement area; capturing at least one image of the measurement areawith the imaging device; transmitting, with the data transmittingassembly, the at least one image to the ground station; measuring, withthe imaging assembly, a distance between the imaging device and theportion of the measurement area; transmitting, with the datatransmitting assembly, the distance between the imaging device and theportion of the measurement area to the monitoring assembly; and scaling,with the scaling unit, the at least one image to determine thegeo-reference data for the portion of the measurement area using theratio stored in the scaling unit; positioning the imaging device in anadir position and capturing the at least one image includes capturingat least one plan view of the measurement area. determining a velocityof an item based on a determining a travel distance the item traveledwithin the measurement area during a time between capturing a firstimage and capturing a second image, wherein said capturing at least oneimage of the measurement area further includes capturing said firstimage of the item within the measurement area and capturing a secondimage of the item within the measurement area; determining a relativedistance between a first and second point within the measurement areascomprising identifying a first portion of the measurement area,identifying a second portion of the measurement area, and scaling the atleast one image to determine the relative distance between the first andsecond portions.
 2. The method of claim 1, wherein measuring thedistance between the imaging device and the portion of the measurementarea includes providing a laser of the imaging assembly and directingthe laser directly downward toward the measurement area.
 3. The methodof claim 1, wherein the measurement area is at least one of land andwater.
 4. A measurement system, comprising: a power supply; a groundstation operably coupled to the power supply and having a memoryincluding a scaling unit, a display, device, and an input device; animaging device operably coupled to the ground station via at least oneof a wired cable and a wireless network, and the imaging deviceincluding a camera with a lens, and the camera being positioned in anadir position and configured to capture at least one plan view of ameasurement area, wherein the imaging assembly includes an inertialmeasurement unit (IMU) configured to determine movement of the imagingdevice; a video receiver configured to receive data from the camera; avideo transmitter operably coupled to the ground station and the videoreceiver and being configured to transmit data from the video receiverto the ground station; and an aerial platform configured to be propelledinto the air and transported to the measurement area by propulsionsystem and hover over the measurement area; a global navigationsatellite system (GNSS) positioned on the aerial platform and operablycoupled to the ground station, and the GNSS being configured determinegeo-reference coordinates of the imaging device; a distance measuringdevice operably coupled to the imaging device and configured todetermine a distance between the imaging device and the measurementarea; a data transmitting assembly positioned on the aerial platform,the data transmitting assembly includes a telemetry consolidation unitconfigured to consolidate images from the imaging device and data fromthe IMU and the GNSS, and the data transmitting assembly is configuredto transmit the consolidated data to the ground station; wherein ageodetic position, which is offset from a boresight of the imagingdevice, of the measurement area is determined by the scaling unit basedon the distance between the imaging device and the measurement area, andthe ground station is configured to display an output of the absolutegeodetic position of a portion of the measurement area. wherein themeasurement area is at least one of land and water. wherein the scalingunit includes a predetermined ratio configured to determine a size of afield-of-view (FOV) of the lens based on a distance between the imagingdevice and the measurement area; wherein the distance measuring devicecomprises a laser rangefinder. wherein the scaling unit furtherdetermines a second geodetic position, which is offset from theboresight of the imaging device, within the measurement area is furtherdetermined by the scaling unit based in part on the distance between theimaging device and the measurement area, and the ground station isconfigured to display an output of the absolute geodetic position of aportion of the measurement area and further scaling the at least oneimage to determine a relative distance between the geodetic position andthe second geodetic position; wherein the imaging device furthercaptures a first image of an object within the measurement area andcaptures a second image of the object within the measurement area,wherein said ground unit and scaling device determines a velocity of theobject by determining a travel distance the object traveled within themeasurement area during a time between capturing the first image andcapturing the second image, wherein the geodetic point and the secondgeodetic point are determined in part based on a user selection of afirst and second point within the imaging device's field of view shownin said display.